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The  U.S.  Army  is  developing  an  Output  Oriented  Re¬ 
source  Management  System  (OORMS)  to  compare  the  value 
of  resources  expended  with  those  received  for  all  Army  pro¬ 
grams.  One  of  the  programs  to  be  encompassed  by  OORMS 
is  Real  Property  Management  Activities  (RPMA). 

The  RPMA  program  consists  of  a  wide  range  of  goals 
from  which  it  is  difficult  to  identify  a  single,  quantifiable 
"entity"  that  represents  the  overall  goal.  Thus,  to  enable  a 
realistic  assessment  in  OORMS,  a  performance  index  is 
needed  to  evaluate  the  efficiency  of  RPMA  at  US.  Army 
installations. 

Three  alternative  modeling  techniques  were  considered 
for  potential  use  in  developing  such  an  index.  Data  En¬ 
velopment  Analysis  IDEA)  was  selected  because  of  its 
ability  to  accommodate  multiple  inputs  and  outputs  simul¬ 
taneously  without  requiring  that  weight  and  functional 
relationships  be  specified. 

The  performance  index  was  applied  to  RPMA  and  its  use 
in  several  stages  of  the  resource  management  process  was 
analyzed.  Results  were  distributed  to  managers  at  the  Major 
Command  (MACOM)  headquarters  and  installation  levels 
for  review.  In  general,  DEA  appears  to  be  a  feasible  model¬ 
ing  technique  for  RPMA  performance;  however,  the  method 
needs  refinement  to  enable  better  discrimination  among 
efficient  installations  and  to  optimize  features  of  the  index. 
A  prototype  should  be  developed  and  implemented  at  the 
installations  to  test  different  input  and  output  measures.  {  <■ 
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DATA  ENVELOPMENT  ANALYSIS  AS  A  TOOL  TO  EVALUATE  EFFICIENCY  OF 
ARMY  REAL  PROPERTY  MANAGEMENT  ACTIVITIES  (RPMA)  SPENDING 


1  INTRODUCTION 


Background 

The  U.S.  Army  is  developing  an  Output  Oriented  Resource  Management  System 
(OORMS)  to  compare  the  value  of  resources  expended  with  services  received  for  all 
Army  programs.  According  to  the  Army  Comptroller,  OORMS  is  intended  to  meet  a 
functional  requirement  of  Army  resource  management:  feedback  on  execution  in  terms 
of  outputs  achieved  for  inputs  planned,  programmed,  budgeted,  and  then  used.  Without 
this  essential  feedback,  it  is  difficult  to  make  consistent,  informed  assessments  as  to  how 
well  the  Army  programs  are  being  both  planned  and  executed.  The  current  resource 
management  process  is  missing  this  vital  link.  Decisions  about  Army  programs  and 
alternatives  can  be  improved  measurably  if  feedback  is  made  an  integral  part  of  the 
process. 1 

Included  in  the  OORMS  assessments  will  be  Real  Property  Management  Activities 
(RPMA)  at  Army  installations.  The  goal  of  RPMA  is  to  develop,  operate,  and  maintain 
the  facilities  necessary  for  the  Army  to  accomplish  its  mission  and  provide  a  quality 
working  and  living  environment  for  its  personnel.2  With  such  a  broad,  multi-objective 
goal,  it  is  difficult  to  identify  a  single,  quantifiable  "entity"  capable  of  representing  the 
overall  goal.  In  other  words,  to  be  able  to  quantify  the  degree  of  goal  achieved  by  an 
installation,  it  is  necessary  to  measure  several  different  outputs— each  accounting  for  a 
different  objective— and  to  aggregate  them  properly  so  as  to  represent  the  amount  of 
goal  achieved.  In  addition,  the  aggregate  measure  of  outputs  has  to  be  comparable  to  the 
amount  of  resources  (input)  deployed  so  that  a  single  composite  index  can  represent  the 
efficiency  of  an  installation.  At  present,  no  such  index  has  been  developed  for  RPMA. 


Objective 

The  objective  of  this  research  was  to  develop  an  output-oriented  performance 
measure  index  to  evaluate  the  efficiency  of  RPMA  at  U.S.  Army  installations.  This 
performance  measure  index  should  relate  the  outputs  achieved  by  RPMA  to  the  resource 
(input)  deployed  during  operations;  that  is,  it  should  be  able  to  compare  several  types  of 
output  with  several  inputs  simultaneously.  In  addition,  the  results  of  evaluations  using 
this  single  index  should  assist  in  the  decision-making  process  of  resource  management. 


Approach 

Three  alternative  methodologies  were  considered  for  index  development:  (1)  data 
envelopment  analysis  (DEA),  (2)  ratio  analysis,  and  (3)  regression  analysis.  Mathematical 


1 Output  Oriented  Resource  Management  System,  Handbook  (Office  of  the  Comptroller  of 
the  Army,  June  1986). 

2Real  Property  Management  Activities  (RPMA),  Executive  Summary,  Vol  1  (Department 
of  the  Army  Study  Group,  March  1978). 
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features  of  all  three  methods  were  compared  and  the  method  found  most  appropriate  for 
dealing  with  RPMA  was  selected. 


Input  and  output  measures  of  RPMA  operations  were  defined  and  selected  to  model 
performance.  The  performance  model  was  applied  to  21  U.S.  Army  Forces  Command 
(FORSCOM)  installations  and  the  results  were  presented  to  the  prospective  users  for 
evaluation.  Feedback  from  the  field  was  analyzed  and  used  to  revise  the  performance 
model. 
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2  DEVELOPMENT  OF  AN  EFFICIENCY  INDEX 


Alternative  Methodologies 

Three  different  methodologies  for  establishing  a  performance  index  were  evaluated 
for  potential  application  to  RPMA.  Each  method  has  already  been  used  in  some  organ¬ 
izations  to  evaluate  different  programs. 

Data  Envelopment  Analysis  (DEA) 

DEA  was  introduced  by  Charnes,  et  al.,3  for  measuring  the  efficiency  of  not-for- 
profit  entities.  The  method  has  been  used  to  measure  the  efficiency  of  several  organiza¬ 
tions  such  as  school  systems,4  health  care  organizations,5  Navy  District  recruiting 
offices,6  fighter  wings  of  the  U.S.  Air  Force,7  and  RPMA  in  the  air  training  commands.8 

DEA  is  designed  to  measure  relative  efficiency  among  similar  organizations,  called 
Decision-Making  Units  (DMU),  that  share  the  same  technology  to  gain  similar  achieve¬ 
ments  (outputs)  by  using  similar  resources  (inputs).  In  this  study,  the  DMUs  are  Army 


3 A.  Charnes,  W.  W.  Cooper,  and  E.  Rhodes,  "Measuring  the  Efficiency  of  Decision- 
Making  Units,"  European  Journal  of  Operational  Research,  Vol  2,  No.  6  (November 
1978),  pp  429-444;  A.  Charnes,  W.  W.  Cooper,  and  E.  Rhodes,  "Short  Communication: 
Measuring  the  Efficiency  of  Decision-Making  Units,"  European  Journal  of  Operational 
Research  (1979),  p  331. 

“A.  Bessent  and  W.  Bessent,  "Determining  the  Comparative  Efficiency  of  Schools 
Through  Data  Envelopment  Analysis,"  Educational  Administrative  Quarterly,  Vol  16, 
No.  2  (1980),  pp  57-75;  A.  Charnes,  W.  W.  Cooper,  and  E.  Rhodes,  "Evaluating  Program 
and  Managerial  Efficiency:  An  Application  of  Data  Envelopment  Analysis  to  Program 
Follow  Through,"  Management  Science,  Vol  27,  No.  6  (1981),  pp  668-697;  A.  Bessent, 
W.  Bessent,  A.  Charnes,  W.  W.  Cooper,  and  N.  Thorogood,  "Evaluation  of  Educational 
Program  Proposals  by  Means  of  DEA,"  Educational  Administration  Quarterly,  Vol  19, 
No.  2  (Spring  1983),  pp  82-107. 

SH.  D.  Sherman,  Measurement  of  Hospital  Efficiency  Using  Data  Envelopment  Analysis, 
unpublished  DBA  thesis  (Graduate  School  of  Business,  Harvard  University,  1981). 

6A.  Lewis  and  R.  C.  Morey,  "Measuring  the  Relative  Efficiency  and  Output  Potential  of 
Public  Sector  Organizations:  An  Application  of  Data  Envelopment  Analysis,"  Inter¬ 
national  Journal  of  Policy  Analysis  and  Information  Systems,  Vol  5,  No.  4  (December 
1981). 

7A.  Bessent,  W.  Bessent,  C.  T.  Clark,  and  J.  Elam,  "Constrained  Facet  Analysis,  A  New 
Method  for  Evaluating  Local  Frontiers  of  Efficiency  and  Performance,"  Air  Force 
Journal  of  Logistics  (Summer  1984),  pp  2-8;  C.  T.  Clark,  Data  Envelopment  Analysis  and 
Extensions  for  Decision  Support  and  Management  Planning,  Ph.D.  dissertation  (The 
University  of  Texas  at  Austin,  May  1983). 

8W.  F.  Bowlin,  A  Data  Envelopment  Analysis  Approach  to  Performance  Evaluation  in 
Not-for-Profit  Entities  With  an  Illustrative  Application  to  the  U.S.  Air  Force,  Ph.D. 
dissertation  (The  University  of  Texas  at  Austin,  December  1984);  W.  F.  Bowlin,  Report 
on  Evaluating  the  Efficiency  of  Real  Property  Maintenance  Activities  in  the  Air 
Training  Command  (Air  Force  Institute  of  Technology,  Wright-Patterson  AFB,  OH, 
November  1984). 
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installations  or,  more  specifically,  the  RPMA  organization  at  the  installation  level.  The 
notation  used  to  formulate  DEA  is: 

Let  DMUj?  j  =  l,..,n  be  the  set  of  DMUs  to  be  evaluated. 

Let  Ij;  i  =  l,...,m  be  the  set  of  input  measures  to  be  used  in  the  evaluation. 

Let  Or;  r  =  l,...,s  be  the  set  of  output  measures  to  be  used  in  the  evaluation. 

Let  0^  represent  the  observed  output  vector  for  DMUj,  where: 

0.  =  (015,.,0i;, ..,()•);  0;i  =  the  amount  of  output  ;used  by  DMU  ; 
j  XJ  *J  1  J 

Let  I  j  represent  the  observed  input  vector  for  DMUj,  where: 

1  j  =  *rj  =  amount  input  r  used  by  DMUj 


Using  these  definitions,  DEA  measures  the  efficiency  of  a  DMU  by  evaluating  the 
ratio  of  weighted  outputs  to  weighted  inputs  as  follows: 


Efficiency  of  DMUjQ,  hjQ 


l  U  0  . 
i,  r  rJ° 


m 

I  V.I. 
i-1  1  1 


jo 


[Eq  1] 


where  Ij-0  =  the  amount  of  input  i  used  by  DMU-  and  O  -  =  the  amount  of  output  r  used 
by  DMUjQ. 

In  the  above  ratio,  IjjQ  and  Orj0  are  observed  values  and  therefore  are  constants. 
The  variables  Ur  (one  for  each  output  measure)  and  Vj  (one  for  each  input  measure)  are 
called  "virtual  multipliers,”  and  their  values  are  computed  relative  to  all  DMUj; 
j=l,..,jo,..,n  by  solving  the  following  mathematical  formulation  (Eq  2): 


Maximize: 


l  U  0  . 
r=l  r  rJ° 


h.  = 

jo  m 

I  V.  I.  . 
i=l  1  11 


jo 


y  u  o  . 

r  rj 


Subject  to: 


r=l 


m 

l  V.  I.  . 
i=l  1 


-  <  1 


V. 


j  —  1  T  •  •  »  n 

>  e  >  0 

>  c  >  0 


[Eq  2] 


where  e  is  a  non- Archimedean  constant  that  constrains  Up  and  Vj  to  positive  values. 
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Observe  that  every  DMU  in  the  set  DMU,,j=l,...,n  is  represented  by  a  constraint  in 
the  above  formulation.  Hence,  there  are  s  plus  m  variables  and  n  constraints,  with  n 
being  the  number  of  units  compared. 

The  above  problem  has  * 1  be  formulated  for  each  DMU  in  the  set  DMUj;  j=l,...,n. 
For  each  formulation,  the  set  of  constraints  is  the  same,  whereas  the  objective  function 
represents  the  DMU  being  evaluated. 

Since  the  DMU  being  evaluated,  DMUj0,  is  also  represented  in  the  constraints  with 

less  than  or  equal  to  one  right-hand  side,  the  value  of  the  objective  function 

is  h; _  =  h.*  <  1  with  h.*  =  1  if  DMU;.  is  efficient  relative  to  the  other  DMUs  present 
JO  jo  jo  jo 

in  the  constraints  of  Equation  3. 

Note  that  in  the  above  formulation  (Eq  2),  the  optimal  Us  and  Vs  are  associated 
with  the  DMU  being  evaluated;  hence,  the  optimal  Us  and  Vs  vary  for  each  DMU. 
Furthermore,  they  represent  the  virtual  multipliers  that  provide  the  highest  possible 
rating  for  the  DMU  being  evaluated  while  ensuring  that  such  multipliers  are  also  feasible 
for  the  other  n-1  DMUs.  In  other  words,  the  formulation  ensures  that  the  efficiency 
rating  assigned  to  a  DMU  is  the  best  one  possible,  and  that  no  other  set  of  weights  (Us 
and  Vs)  will  assign  a  higher  efficiency  rate. 

Therefore,  when  the  efficiency  of  DMU^0  is  less  than  1,  it  follows  that  the  jo**1  unit 
is  strictly  inefficient  compared  with  some  other  DMUs  in  the  set.  The  subset  of  DMUs 

against  which  the  joth  unit  is  compared  is  called  the  "joth  unit  reference  set”;  this  subset 
consists  of  DMUs  for  which  the  constraint  is  equal  to  1  at  optimality.  In  addition,  DEA 
results  provide  ways  to  project  the  inefficient  unit  into  its  reference  set  so  that  it 
becomes  efficient. 

The  formulation  given  in  Equation  2  is  an  extended  nonlinear  programming  formula¬ 
tion  of  an  ordinary  fractional  programming  problem.  However,  Charnes,  et  al.9  have 
shown  that  it  can  be  transformed  into  an  equivalent  linear  programming  problem  using 
the  linear  fractional  programming  theory  developed  by  Charnes  and  Cooper. 10  Appendix 
A  explains  the  linear  programming  formulation  and  its  solution. 

Ratio  Analysis 

This  approach  has  been  used  widely  to  measure  performance  in  almost  every  type 
of  organization.  The  performance  measure  is  determined  by  evaluating  the  ratio 
between  a  weighted  sum  of  inputs  and  a  weighted  sum  of  outputs.  To  do  this,  the  weights 
are  predetermined— not  calculated  as  in  DEA.  This  method  is  well  suited  for  rating 
among  units  in  which  inputs  and  outputs  can  be  quantified  using  the  same  measuring  unit 
(e.g.,  U.S.  dollars). 


9A.  Charnes,  W.  W.  Cooper,  and  E.  Rhodes  (1978);  A.  Charnes,  W.  W.  Cooper,  and  E. 
Rhodes  (1979). 

10 A.  Charnes  and  W.  W.  Cooper,  "Programming  With  Linear  Fractional  Functionals," 
Naval  Research  Logistics  Quarterly,  Vol  9  (1962). 
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When  Inputs  and  outputs  do  not  share  the  same  measuring  units,  it  can  become 
difficult  to  find  a  meaningful  performance  ratio.  Furthermore,  the  complexity  of  the 
problem  increases  when  the  relationships  between  input  and  output  are  unknown. 

Several  relatively  simple  ratios  might  be  used  to  model  single  relationships  between 
different  sets  of  inputs  and  outputs.  However,  the  separate  ratios  do  not  explicitly 
recognize  the  joint  use  of  these  inputs  to  produce  a  combination  of  outputs.  Therefore,  a 
unit  may  be  rated  highly  efficient  based  on  one  of  these  ratios,  while  the  same  unit  may 
be  rated  inefficient  with  respect  to  a  different  ratio.  This  situation  leads  to  some 
ambiguity  as  to  whether  a  unit  is  efficient  or  inefficient,  and  calls  for  a  method  of 
establishing  different  priorities  (weights)  among  the  separate  ratios  to  generate  an 
overall  efficiency  ratio. 

However,  when  dealing  with  a  few  relevant  outputs  and  the  priorities  among  them 
are  clear,  ratio  analysis  allows  management  to  stress  the  goals  of  the  organization  more 
effectively  than  with  DEA.  Furthermore,  ratio  analysis  provides  a  way  to  penalize 
deviation  in  the  inputs  or  in  the  outputs  from  some  preestablished  targets.  Also,  it 
permits  management  to  change  the  priorities  of  the  goal  from  time  to  time  when  the 
needs  of  the  organization  call  for  it. 

The  weaknesses  of  ratio  analysis  are  that:  (1)  when  aggregating  many  inputs  and 
outputs,  the  index  loses  meaning,  (2)  goals  that  do  not  show  in  the  index  are  disregarded 
by  lower  level  management,  and  (3)  the  design  of  the  ratio  may  need  to  be  rather 
complex  to  avoid  ambiguity. 

Regression  Analysis 

This  approach,  although  not  as  widely  used  as  ratio  analysis,  is  very  popular  among 
many  organizations  for  estimating  relationships  between  one  input  and  one  output  (linear 
regression).  Regression  analysis  usually  performs  well  when  used  to  evaluate  systems  of 
one  input  and  a  few  outputs  or  vice  versa— one  output  and  several  inputs.  The  approach 
can  be  viewed  as  a  technique  to  design  a  ratio,  in  which  case  regression  analysis 
simplifies  the  modeling  of  unknown  relationships  among  inputs  and  outputs.  On  the  other 
hand,  regression  analysis  does  not  allow  managers  to  emphasize  selected  goals  and 
targets  over  the  others. 


Methodology  Selection 

Upon  analyzing  features  of  the  three  methods,  it  became  clear  that  DEA  offered 
several  advantages  over  the  other  two  methods  for  evaluating  the  efficiency  of  RPMA. 
These  advantages  are  summarized  here;  for  a  more  extensive  comparison,  refer  to 
Bowlin,  et  al. 1 1  and  Banker,  et  al. 1 2 


1  'W.  F.  Bowlin,  A.  Charnes,  W.  W.  Cooper,  and  H.  D.  Sherman,  A  Comparative  Study  of 
Data  Envelopment  Analysis  and  Other  Approaches  to  Efficiency  Estimation,  Research 
Report  CCS  451  (Center  for  Cybernetic  Studies,  The  University  of  Texas  at  Austin, 
September  1983). 

l2R.  D.  Banker,  R.  F.  Conrad,  and  R.  P.  Strauss,  "A  Comparative  Application  of  Data 
Envelopment  Analysis  and  Translog  Methods:  An  Illustrative  Study  of  Hospital  Produc¬ 
tion,”  Management  Science,  Vol  32,  No.  1  (January  1986),  pp  30-44. 
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DEA  differs  from  ratio  analysis  in  that  it  handles  multiple  inputs  and  outputs 
simultaneously  without  requiring  a  priori  specification  of  weights.  Moreover,  the 
production  function,  i.e.,  efficient  input-output  relationship,  need  not  be  known  to 
evaluate  efficiency  with  DEA. 

DEA  differs  from  regression  analysis  and  related  statistical  techniques  in  that  it  is 
nonparametric  and  thus  does  not  require  specification  of  the  functional  forms  and 
relations  to  be  used.  In  addition,  DEA  evaluates  the  efficiency  of  a  DMU  against  the 
most  efficient  DMUs  (i.e.,  the  efficiency  "frontier”)  and  not  against  interior  DMUs 
(average). 

There  are,  however,  two  limitations  to  DEA  that  should  be  mentioned: 

•  DEA  will  not  necessarily  locate  all  inefficient  units;  in  other  words,  DEA 
measures  only  relative  inefficiency. 

•  DEA  is  capable  of  addressing  efficiency  only  and  does  not  attempt  to  evaluate 
the  effectiveness  of  the  inputs  used  and/or  the  outputs  obtained. 


DEA  Application  Process 

The  DEA  method  is  implemented  in  three  steps:  (1)  identification  of  units  to  be 
evaluated,  (2)  identification  of  input  and  output  measures,  and  (3)  application  of  DEA  and 
analysis  of  results. 

Step  1:  Identification  of  Units  To  Be  Evaluated 

Managers  should  identify  the  organizational  units  for  which  a  DEA  efficiency 
evaluation  would  be  of  interest.  Generally,  this  parameter  would  be  a  set  of  units  that 
provide  similar  services  and  whose  performance  management  wants  to  evaluate  to 
improve  efficiency.  For  this  study,  the  units  to  be  evaluated  are  the  RPMA  organiza¬ 
tions  at  the  installation  level. 

Step  2:  Identification  of  Input  and  Output  Measures 

Managers  should  identify  the  relevant  inputs  and  outputs  of  units  to  be  evaluated  as 
measured  over  a  representative  period  of  time  (e.g.,  for  1  year,  quarter,  or  month). 
Relevant  outputs  are  those  services  and  other  activities  for  which  the  unit  is  responsible 
in  achieving  its  mission  (i.e.,  goals).  Relevant  inputs  are  the  resources  required  to 
produce  the  designated  outputs. 

Field  applications  of  DEA  have  indicated  that  this  process  of  output  and  input 
identification  often  is  by  itself  useful  to  managers,  as  outputs  and  inputs  sometimes  are 
not  explicitly  identified  or  understood.  In  addition,  some  of  the  relevant  outputs  and 
inputs  may  not  have  been  measured  or  captured  in  the  current  information  system.  The 
absence  of  data  on  relevant  outputs  and  inputs  has  raised  questions  about  the  adequacy  of 
the  information  system,  since  this  type  of  input/output  data  is  needed  to  assess  operating 
performance  regardless  of  the  techniques  used. 
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Unless  all  relevant  outputs  and  inputs  are  included  in  the  DEA  analysis,  the  DEA 
results  will  have  to  be  reviewed  for  any  bias  that  might  result.  There  are  four  general 
guidelines  for  selecting  the  input  and  output  measures: 

1.  The  inputs  and  outputs  should  be  comprehensive.  That  is,  they  should  fully  and 
properly  measure  the  RPMAs. 

2.  There  should  be  some  basis  for  believing  that  the  relationship  between  inputs 
and  outputs  is  such  that  an  increase  in  an  input  can  reasonably  be  expected  to  increase 
one  or  more  of  the  outputs. 

3.  All  input  and  output  measures  should  exist  in  positive  amounts  for  each 
installation. 

4.  The  variables  should  be  identifiable  and  defined  and  controlled  so  that  they 
cannot  be  manipulated  in  reports,  or  at  least  the  resulting  data  should  be  reviewed  to 
remove  these  effects  which  might  otherwise  influence  the  results  of  the  performance 
model.  As  discussed  earlier,  DEA  not  only  rates  efficiency  among  installations,  but  also 
provides  additional  information  regarding  the  efficiency  of  input  usage  and  output 
achievement.  Therefore,  the  choice  of  inputs  and  outputs  will  determine  the  value  of 
information  provided  by  the  model.  In  other  words,  a  choice  of  meaningless  input  and 
output  measures  will  yield  meaningless  managerial  information.  Hence,  to  fully  exploit 
the  capabilities  of  DEA,  two  more  guidelines  were  used  for  selecting  input  and  output 
measures  in  this  particular  application: 

5.  The  level  of  detail  in  defining  inputs  and  outputs  should  be  sufficient  for  Office 
of  the  Chief  of  Engineers  (OCE)-level  planning,  programming,  and  budgeting  activities. 

6.  Existing  data  sources  should  be  used  whenever  possible,  rather  than  generating 
new  data. 

Step  3:  Application  of  DEA  and  Analysis  of  Results 

At  this  point,  the  results  of  DEA  may  not  be  consistent  with  prior  perceptions  of 
the  units  evaluated.  This  outcome,  in  turn,  may  raise  questions  about  how  complete  and 
representative  the  output  and  input  measures  are.  In  this  way,  the  analysis  may  result  in 
refinements  to  the  set  of  inputs  and  outputs  used  in  the  model. 


Identification  of  Input  Measures 

In  general  terms,  there  are  two  inputs:  monetary  funds  and  space  used  for 
supporting  RPMA.  Monetary  funds,  as  a  single  input  term,  may  not  serve  the  purposes  at 
the  OCE  level.  Two  alternative  disaggregation  schemes  are  possible: 

1.  According  to  the  destination  of  the  funds  (labor,  materials,  utilities,  and 
equipment),  or 

2.  According  to  the  current  accounting  system  (J,  K,  L,  and  M  accounts). 
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Furthermore,  the  level  of  disaggregation  can  be  considered  at  different  details 
(e.g.,  type  of  labor,  or  J  ,  J2).  Given  that  the  main  purpose  of  performance  measure¬ 
ment  is  planning,  programming,  and  budgeting  at  the  OCE  level,  the  following  input  list 
is  recommended  (Figure  1): 

I  :  Cost  of  operation  of  utilities:  J  account. 

I2:  Cost  of  M<5cR  of  real  property:  K  account. 

I3:  Cost  of  minor  construction:  L  account. 

I4:  Cost  of  engineering  support:  M  account. 

ls:  Square  feet  of  maintenance  and  production  buildings. 

Identification  of  Output  Measures 

In  general  terms,  the  RPMA  resources  are  used  to: 

1.  Accommodate  and  provide  services  to  Army  personnel. 

2.  Maintain  and  repair  facilities. 

Therefore,  outputs  can  be  divided  into  two  major  groups:  those  related  to  personnel  and 
those  related  to  the  physical  plant.  This  division  is  shown  in  Figure  2  and  is  explained 
below. 

Personnel  Related  Outputs 

Quantity  as  well  as  quality  of  the  service  provided  to  persons  living  and  working  at 
the  installations  should  be  considered.  Therefore,  the  following  outputs  are  proposed: 

Ot:  population  served;  measures  the  quantitative  aspect  of  serving  people. 

02:  (population  served)/(number  of  complaints  +  1);  measures  the  quality  of  service 
provided  to  the  people. 

Note  that,  as  the  number  of  complaints  decreases,  02  increases.  Hence,  02  can  be 
perceived  as  a  surrogate  measure  for  the  quality  of  life  at  an  installation. 

Facility  (Physical  Plant)- Related  Outputs 

An  Army  installation  does  not  have  an  external  output.  Moreover,  internal  outputs 
are  difficult  to  measure,  as  with  any  nonprofit  organization. 

In  a  resource  management  system,  one  way  of  determining  the  outputs  is  to  answer 
this  question:  for  what  purposes  are  the  resources  used  at  an  installation?  The  physical 
facilities  use  the  inputs  (resources)  listed  above  in  two  ways: 

1.  As  space  and  surface  that  require  maintenance,  repair,  and  upgrading. 

2.  As  space  that  consumes  energy. 


J  ACCOUNT  K  ACCOUNT  L  ACCOUNT  M  ACCOUNT  SPACE 


Figure  1.  RPMA  inputs. 


OUTPUTS 


QUANTITY  QUALITY  MAINTENANCE  8  REPAIR  ENERGY  CONSUMPTION 


QUANTITY  QUALITY  QUANTITY  QUALITY 


Figure  2.  RPMA  outputs. 
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The  amount  of  space  and  surface  maintained,  repaired,  and  upgri^d  is  an  output; 
likewise,  the  quality  of  these  services  is  an  output,  as  is  the  amount  o’  space  that 
consumes  energy.  However,  the  quality  of  energy  consumption  is  not  considered  here, 
since  it  is  an  output  of  a  different  Army  program~the  Energy  Conservation  Improvement 
Program— (ECIP),  not  of  RPMA. 

The  relationship  between  the  outputs  and  inputs  proposed  can  vary  substantially 
from  one  facility  to  another.  The  following  features  of  the  facilities  introduce  this 
variance: 

•  Function  (e.g.,  warehouse  versus  hospital), 

•  Structural  characteristics  (e.g.,  paved  surface  versus  building), 

•  Geographic  location. 

Therefore,  the  specific  definitions  of  the  outputs  should  account  for  these  features.  In 
view  of  these  considerations,  the  following  outputs  are  proposed: 

03:  Cost  of  duplicating  the  existing  facility.  This  duplication  cost  (DC)  can  be 
computed  as  follows: 1 3 

DC  =  Acquisition  Cost  x  Index  (1)  +  Historical  Renewal  Costs  x  Index  (2). 

Renewal  includes  improvements  in  the  existing  facility,  but  excludes  main¬ 
tenance  and  repair.  Indexes  1  and  2  can  be  obtained  from  the  Building 
Construction  Index  published  by  the  Engineering  News  Record. 

Olf:  Total  of  backlog  maintenance  and  repair  (BMAR)  and  deferred  maintenance 
and  repair  (DMAR).  This  output  is  undesirable;  therefore,  the  inverse, 
1/(BMAR  +  DMAR),  should  be  used  in  the  performance  formula. 

Os:  Number  of  square  feet  of  building  space  conditioned;  modified  by  a  building 
type  energy  use  factor14  and  by  heating/cooling  degree  days. 

The  output  O  represents  the  "quantity"  of  the  physical  facilities  maintained  and 
repaired;  it  accounts  for  the  functional,  structural,  and  locational  differences  between 
the  facilities.  Furthermore,  03  is  directly  related  to  the  inputs  proposed  under  Identifi¬ 
cation  of  Input  Measures.  Therefore,  it  is  a  more  useful  measure  than  square  feet  of  the 
facilities  maintained  and  repaired. 

The  BMAR  and  DMAR  values  in  output  04  are  commonly  used  as  surrogate 
measures  of  the  facilities'  physical  condition.  They  also  account  for  the  functional, 
structural,  and  locational  differences  among  facilities.  Therefore,  in  the  absence  of  a 
more  direct  measure  for  the  condition  of  the  facilities,  can  be  used  as  a  surrogate 
measure  for  the  quality  of  maintenance  and  repair  consumption. 


130.  Coskunoglu,  "Appraisal  of  Army's  Facilities,"  RPMA  Information  Paper  No.  37  (U.S. 

Army  Construction  Engineering  Research  Laboratory,  November  1985). 

14  Developed  by  the  U.S.  Army  Construction  Engineering  Research  Laboratory  (USA- 
CERL);  see  Design  Criteria ,  Architectural  and  Engineering  Instructions  (OCE, 
Engineering  Division,  13  March  1987).  Available  through  PAXMAIL. 
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Finally,  0$  is  a  measure  of  output  pertaining  to  energy  consumption.  Again,  it 
accounts  for  the  functional,  structural,  and  locational  differences  among  facility  types. 

Data  Availability 

To  avoid  having  to  generate  new  data,  a  compromise  was  reached  between 
desirable  data  and  data  on-hand.  As  a  result,  the  list  of  inputs  and  outputs  used  in  this 
study  can  be  summarized  as: 

O  :  Number  of  people  living  and  working  at  the  installation. 

02:  Not  used. 

03:  Redefined  as  millions  of  square  feet  of  building  area. 

O^:  109/BMAR;  only  BMAR  is  used. 

Os:  (building  area  in  sq  ft)  x  (USA-CERL  energy  factor  for  building  type  and 
location). 

Total  J,  K,  L,  and  M  accounts,  respectively. 

I5:  Total  area  of  the  buildings  (sq  ft)  used  for  maintenance  and  repair 
purposes. 
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3  DEA  COMPUTATIONS  AND  RESULTS 


Computations 

A  portable  personal  computer  (Compaq  286)  with  an  off-the-shelf  linear  program¬ 
ming  package  (Vino)  was  used  to  solve  the  DEA  computations.  Vino  was  chosen  because 
it  reads  data  from  Lotus  1-2-3,  which  is  the  most  popular  spreadsheet  program. 

Results  were  computed  for  two  major  commands  (MACOMs)  over  3  fiscal  years  as 
follows: 

1.  MACOM  #1,  FY85 

2.  MACOM  #1,  FY84 

3.  MACOM  #1,  FY83 

4.  MACOM  #2,  FY85 

5.  MACOMs  #1  and  #2,  FY85;  joint  analysis. 

The  results  for  each  computation  are  explained  in  Appendix  B.  Table  1  summarizes  the 
results  of  MACOM  #1  for  FY85. 


Table  1 

Summary  of  Results  for  MACOM  #1  in  FY85 


Instal lation 

Efficiency 
Rate  (S) 

Reference 

Set* 

No.  of  Times  in 

Reference  Set** 

1 

100.00 

1 

2 

2 

100.00 

2 

0 

3 

100.00 

3 

4 

4 

100.00 

4 

0 

5 

95.13 

10,15,20,21 

- 

6 

93.74 

1,12,14,21 

- 

7 

87.86 

3,10,14,20 

- 

8 

99.11 

1,14,21 

- 

9 

100.00 

9 

2 

10 

100.00 

10 

7 

11 

100.00 

11 

1 

12 

100.00 

12 

1 

13 

69.67 

10,15,20,21 

- 

14 

100.00 

14 

3 

15 

100.00 

15 

5 

16 

75.15 

3,9,10,15,20 

- 

17 

55.11 

10,15,20 

- 

18 

89.37 

3,9,10,15,20 

- 

19 

65.89 

3,10,11 

- 

20 

100.00 

20 

6 

21 

100.00 

21 

4 

•The  set  of  100  percent  efficient  installations  against  which  the  inefficient 
installation  is  compared.  Also  see  the  section  Alternative  Methodologies  and 
Appendix  A. 

••The  more  times  an  installation  is  in  a  reference  set,  the  more  indisputable 
its  efficiency.  Furthermore,  when  an  installation  is  rated  100  percent  and 
it  is  not  In  any  reference  set,  Its  efficiency  can  be  questioned. 


Installations  within  MACOM  #1  that  were  rated  100  percent  efficient  in  each  of 
the  3  years  examined  were  numbers  3,  9,  10,  11,  12,  15,  and  20.  In  contrast,  the  set  of 
installations  rated  as  less  than  100  percent  efficient  in  each  of  those  same  3  years 
included  numbers  6,  13,  16,  and  18. 

To  further  review  and  validate  the  results  obtained  for  MACOM  #1  from  this 
analysis,  data  were  gathered  for  FY85  from  21  installations  in  MACOM  #2  and  combined 
with  those  of  MACOM  #1  for  a  joint  analysis.  With  this  approach,  more  observations 
could  be  included  in  the  analysis  which  should  provide  better  overall  efficiency  in  the 
evaluations.  Even  if  the  two  sets  of  installations  are  not  quite  comparable,  it  should 
appear  in  the  results  as  a  consistent  separation  of  their  efficiency  evaluations. 

Tables  2  and  3  compare  the  efficiency  rating  calculated  using  FY85  annual  data 
when  the  MACOMs  were  analyzed  separately  (column  2)  and  when  they  were  analyzed 
together  (column  3). 


Table  2 


Comparison  of  Efficiency  Ratings— MACOM  #1  and 
Joint  Analysis  (FY85) 


Instal lation 

MACOM  *1 

MACOMs  #1  &  #2 

1 

100.00 

100.00 

2 

100.00 

98.38 

3 

100.00 

98.69 

4 

100.00 

100.00 

5 

95.13 

95.13 

6 

93.75 

88.35 

7 

87.75 

83.33 

8 

99.12 

93.57 

9 

100.00 

100.00 

10 

100.00 

100.00 

11 

100.00 

93.44 

12 

100.00 

100.00 

13 

69.67 

69.67 

14 

100.00 

100.00 

15 

100.00 

100.00 

16 

75.15 

71.07 

17 

55.11 

48.20 

18 

89.38 

80.83 

19 

65.90 

47.28 

20 

100.00 

100.00 

21 

100.00 

100.00 

20 


Table  3 


Comparison  of  Efficiency  Ratings— MACOM  #2 
and  Joint  Analysis  (FT85) 


Installation  MACOM  §2  MACOMs  /l  A  K 


1 

100.00 

100.00 

2 

99.89 

97.42 

3 

91.02 

87.48 

4 

100.00 

100.00 

5 

100.00 

100.00 

6 

100.00 

100.00 

7 

79.17 

78.66 

8 

100.00 

100.00 

9 

100.00 

100.00 

10 

79.04 

71.97 

It 

100.00 

100.00 

12 

100.00 

too. 00 
'95. 9  \ 

13 

100.00 

14 

100.00 

100.00 

15 

100.00 

100.00 

16 

90.48 

89.65 

17 

78.17 

70.11 

18 

100.00 

100.00 

19 

62.60 

61.87 

20 

87.74 

85.10 

21 

100.00 

100.00 

Validation  of  Results 

The  results  were  submitted  to  prospective  users  of  RPMA  resources  for  validation 
at  two  different  levels:  (1)  OCE  and  (2)  the  installations.  At  the  headquarters  level,  the 
managers  agreed  that,  in  the  case  of  MACOM  #1,  the  installations  identified  by  DEA  as 
inefficient  were  likely  to  be  relatively  inefficient,  although  values  of  the  slacks* 
associated  with  the  inputs  and  outputs  in  some  of  the  cases  were  viewed  as  out  of 
range.  In  addition,  the  managers  thought  that  the  efficiency  rate  assigned  to  installation 
#17  (55.11  percent)  was  too  low  and  might  be  due  to  the  geographic  location  of  the 
installation  (Alaska). 

Regarding  the  results  for  MACOM  #2,  managers  at  the  headquarters  level  strongly 
agreed  that  installation  #19,  which  was  rated  at  62.6  percent  in  FY85,  is  one  of  the  most 
efficient  installations.  In  both  cases  (i.e.,  MACOM  #1  and  MACOM  #2),  the  managers 
agreed  that  not  all  of  the  installations  rated  as  100  percent  efficient  could  be  perceived 
as  equally  efficient.  In  other  words,  the  managers  held  that  the  model  should  be  able  to 
differentiate  better  among  efficient  installations. 


*A  geometrical  interpretation  of  the  slacks  variables,  along  with  their  suggested  usage 
by  managers,  is  given  in  Appendix  A. 
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At  the  Installation  level,  as  might  be  expected,  managers  were  less  receptive  of  the 
model's  capabilities.  From  their  viewpoint,  the  model  suggests  budget  reductions  since 
the  level  of  outputs  is  not  under  their  control.  However,  the  installation  managers 
offered  valuable  suggestions  toward  improving  the  model. 

Analysis  of  the  user  feedback  from  both  levels  reveals  that  the  model  should 
produce  better  results  with  the  following  adjustments: 

1.  The  replacement  value,  as  defined  earlier,  should  be  used  as  03,  instead  of 
square  feet  of  building,  to  help  account  for  the  variety  of  building  types. 

2.  Instead  of  using  the  inverse  of  BMAR  as  an  output,  BMAR  should  be  considered 
an  input. 


3.  I  ,  I2,  I3,  and  I4  should  be  aggregated  into  only  one  input,  Ij  =  $(J+K+L+M). 

The  advantage  of  the  first  recommendation  was  explained  in  Chapter  2.  To  under¬ 
stand  how  the  second  recommendation  will  benefit  the  model,  consider  installation  #19 
and  its  reference  set  for  the  analysis  of  MACOM  #2  in  FY85.  Table  4  shows  the  results 
of  evaluating  those  installations  under  10  different  criteria.  Clearly,  installation  #19  is 
less  efficient  for  9  of  the  10  criteria.  Only  under  the  criterion  BMAR/Ksq  ft  is  installa¬ 
tion  #19  the  most  efficient.  Note  also  that  the  seven  criteria  under  which  installation 
#19  is  inefficient  are  all  included  in  the  performance  model,  whereas  BMAR/Ksq  ft  is 
not.  For  this  criterion  to  be  included  in  the  performance  model,  BMAR  has  to  be 
considered  an  input.  As  such,  the  model  would  rate  installation  #19  as  100  percent 
efficient  since  no  other  installation  is  more  efficient  under  this  criterion. 

Finally,  the  third  recommendation  will  improve  the  quality  of  the  results  because 
the  number  of  100  percent  efficient  installations  will  decrease  as  aggregation  reduces 
the  number  of  degrees  of  freedom  for  the  model.  In  addition,  aggregation  of  the  four 
accounts  into  only  one  input  (I  )  will  avoid  the  difficulties  in  explaining  the  value  of  the 
slacks  associated  with  each  account. 
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Table  4 


Comparison  of  Installation  #19  With  Its  Reference  Set 


Cr i ter  1  a 

Instal lation 

SK/Ksq  ft 

JL/Ksq  ft 

$M/Ksq  ft 

5 

$576.85 

$94.6 

$248.10 

11 

$2,432.00 

$318.00 

$953.33 

12 

$1,314.19 

$200.27 

$827.72 

18 

$1,074.79 

$67.55 

$424.31 

19 

$2,075.79 

$445.05 

$1 ,478.54 

$(J*-K+UM)/Pop. 

$(  J+K+L+M)  Asq  ft 

$BMARAsq  ft 

5 

$3,061.84 

$1 ,066.90 

$2,388.61 

11 

$268.13 

$4,048.67 

$800.73 

12 

$1,577.86 

$3,659.35 

$679.21 

18 

$2,204.70 

$2,010.55 

$1 ,355.82 

19 

$2,108.26 

$5,152.20 

$201.19 

SM/Pop. 

SJ/Pop. 

5 

$712.01 

$422.85 

11 

$56.07 

$20.31 

12 

$388.77 

$477.75 

18 

$465.28 

$486.77 

19 

$605.01 

$471.73 

V'5 

$J/Btu 

5 

5.3432 

$0.0023 

11 

13.0435 

$0.0058 

12 

36.1590 

$0.0145 

18 

9.2885 

$0.0081 

19 

9.2667 

$0.0179 

Oj  =  Replacement  value 
1^  =  Sq  ft  of  shops 


23 


4  IMPLICATIONS  FOR  MANAGEMENT 


DEA  was  next  evaluated  for  use  in  the  following  managerial  tasks:  audit  and 
review  of  operations,  planning,  and  budgeting,  and  resources  determination  and  predic¬ 
tion.  Results  from  the  previous  analysis  were  applied  to  this  evaluation  to  study  the 
model  from  a  practical  standpoint. 


DEA  as  a  Management  Audit  Tool 

The  most  straightforward  use  of  the  DEA  results  is  for  auditing  to  distinguish 
between  relatively  efficient  and  inefficient  installations.  By  doing  so,  managers  can 
concentrate  on  those  inefficient  installations  most  likely  to  profit  from  further 
analysis. 15  In  addition,  since  the  efficient  installations  are  in  the  reference  set  of 
inefficient  installations,  managers  can  make  comparisons  to  better  understand  the  source 
of  inefficiency.  In  comparing  an  inefficient  installation  with  its  reference  set,  the  values 
of  the  slacks  for  inputs  and  outputs  show  where  the  comparison  is  weaker  and  by  how 
much.  Therefore,  by  analyzing  these  values,  managers  should  be  able  to  understand  the 
particulars  of  the  inefficient  installation  and  whether  those  properties  are  admissible. 

Efficient  installations  also  have  to  be  analyzed  since  DEA  results  measure  only 
relative  efficiency.  The  values  of  the  "virtual  multipliers"  (U0,  Vt),  provide  information 
about  the  input  and  output  mixture  that  rates  these  installations  as  100  percent  effi¬ 
cient.  Thus,  managers  have  to  decide  if  that  mixture  is  the  appropriate  one. 


Planning  and  Budgeting  With  DEA 

For  managers  at  installations  rated  less  than  100  percent  efficient,  there  is  an 
immediate  way  to  profit  from  the  performance  model  results:  by  planning  in  such  a  way 
as  to  become  more  like  the  units  in  their  reference  set  (hence,  more  efficient). 

Another  possible  use  of  the  performance  model  is  in  the  budgeting  process.  At  the 
MACOM  level,  the  performance  model  can  be  used  to  simulate  the  budgets  submitted  by 
each  installation  as  if  they  were  real  data.  For  the  installations  whose  budgets  are  rated 
as  less  than  100  percent  efficient,  MACOM  management  can  "negotiate"  an  alternative 
budget  based  on  the  budgets  of  the  other  installations  in  their  reference  set.  For  the 
installations  whose  budgets  are  rated  as  100  percent  efficient,  MACOM  managers  should 
ensure  that  (1)  the  budget  is  appropriate  and  (2)  the  installations  follow  it  as  closely  as 
possible. 

As  an  example  of  using  the  performance  model  in  budgeting,  suppose  that  the 
requested  budget  of  each  installation  in  MACOM  #2  is  that  shown  in  Table  5.  Table  6 


l5H.  D.  Sherman,  "Data  Envelopment  Analysis  as  a  New  Managerial  Audit  Methodology- 
Test  and  Evaluation,"  Auditing:  A  Journal  of  Practice  and  Theory  Vol  4,  No.  1  (Fall 
1984),  pp  35-53. 


Table  5 


Example  Requested  Budgets  for  MACOM  #2  Installations 


Replacement 


Instal¬ 

lation 

Popu 1  at i on 
Served 

Value 

D.C. 

Energy 

Budget 

$( J+K+L+M) 

BMAR 

Msg  ft 

1 

8.904 

1.750433 

0.693742 

54.135 

15.1964 2 

0.596 

2 

46.049 

2.893076 

1 .525868 

74.879 

14.95708 

1 .42  6 

3 

42.212 

2.257709 

1 .310005 

56.891 

17.57870 

1.575 

4 

2.265 

1 .979978 

0.099975 

6.37 

1.141 

0.025 

5 

1.698 

0.358625 

0.316338 

5.199 

1 1  .6397 

0.912 

6 

46.233 

1 . 430648 

0.716229 

61 .64 

5.3088 

0.302 

7 

15.765 

1 .62362 

0.495335 

32.464 

1 1 .991 

0.449 

8 

23.874 

1 .073542 

0.615065 

35.712 

5.517 

0.366 

9 

2.784 

0.965905 

0.254581 

13.458 

0.89 

0.157 

10 

12.656 

0.695382 

0.378755 

29.659 

3.809 

0.362 

1 1 

25.503 

0.091396 

0.08973 

6.073 

1.2011 

0.115 

12 

25.174 

1 .251925 

0.827437 

39.721 

8.031 

0.327 

13 

40.012 

2.61558 

1 .346354 

60,181 

28.54462 

1  .392 

14 

15.499 

0.753619 

0.46455 

23.434 

9.509921 

0.257 

15 

15.822 

0.756672 

0.473115 

40.136 

26.898 

0.207 

16 

19.015 

1 .08138 

0.36737 

29.833 

1.321 

0.242 

17 

3.807 

0.493099 

0.132355 

12.258 

2.29 

0.096 

18 

3.024 

1  .549681 

0.18198 

6.667 

4.495892 

0.357 

19 

19.702 

0.959136 

0.519221 

41.537 

1.622 

0.87 

20 

35.387 

2.815029 

0.95207 

52.144 

5.288 

1.121 

21 

47.023 

2.796394 

0.842773 

36.66 

2.281 

0.651 

lists  the  results  of  applying  the  performance  model  to  these  requested  budgets.  The 
results  in  terms  of  installation  #1  can  be  interpreted  as  follows: 

•  The  budget  requested  by  Installation  #1  is  only  60.07  percent  efficient  com¬ 
pared  with  the  budgets  requested  by  the  rest  of  the  installations  in  MACOM  #2. 

•  Installations  that  had  the  most  influence  on  installation  #  l's  efficiency  rate  are 
4,  12,  and  21  (reference  set). 

To  increase  the  efficiency  of  installation  # l's  budget,  management  should  try  to 
decrease  the  amount  of  inputs  budgeted.  As  an  example,  there  are  three  possible  ways  of 
making  this  installation's  requested  budget  100  percent  efficient: 

1.  Reduce  (J+K+L+M)  by  47  percent,  or 

2.  Reduce  BMAR  by  88  percent,  or 

3.  Reduce  (J+K+L+M)  and  BMAR  by  46  percent  each. 
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Table  6 

Example  Results  of  Applying  Performance  Model  to  Requested  Buckets 


Table  6  (Cont'd) 


0,036)64  0  0.0001  0.0001  197.5519  3.749458  0.0001  0.0001  0.0001  47.21495 

15.499  0  0.756619  0  0.46455  23.434  9.509921  0  0  0.257  92.33 

0  0  0.062790  0  0  0  4.413071  0  0  0 


Resource  Determination  With  DEA 

One  of  the  most  attractive  features  of  DEA  is  its  ability  to  identify  the  efficiency 
frontier,  which  allows  managers  to  know  if  a  "production  possibility"*  will  be  relatively 
efficient  compared  with  the  rest  of  the  DMU.  This  property  of  DEA  can  be  exploited 
further  by  managers  when  trying  to  predict  the  level  of  resources  (inputs)  needed  to 
reach  a  specific  level  of  outputs  in  the  future. 

The  above  problem  can  be  approached  using  DEA  as  follows: 

Let  0d  =  (C>1(j,...,Og£j)  be  the  known  level  of  outputs  that  installation  d  wants  to 
produce  next  year. 


Let  ld  =  (I!(j,...,Im(j)  &e  the  unknown  input  vector  necessary  for  installation  d  to 
produce  Od  next  year. 

Let  7  =  (I,  _,..., I__)  be  an  estimated  input  vector  for  installation  d.  In  other 

_  1 5  ^*o  _ 

words,  I  is  an  input  vector  with  which  management  "feels”  confident  to  produce  Od 
next  year  in  installation  d.  Then: 


Let  DMUd  be  a  dummy  DMU  with  its  input-output  vector  represented  by  (0d,  Ig) . 

Now  DMUD  can  be  added  to  the  set  of  installations,  DMUjj  j=l,...,n,  and  the 
efficiency  of  DMUg  can  be  evaluated  with  respect  to  the  extended  set  of  installations, 
DMUj;  j=l,...,n,D. 

Suppose  that  DMUD  is  rated  as  less  than  100  percent  efficient.  Then,  project¬ 
ing  <0  ,  7  )  into  the  efficiency  frontier16  will  make  DMUjj  100  percent  efficient.  In 
other  wortis,  the  input  vector  =  e  •  I  -  sL  will  be  enough  to  produce  next  year 
if  DMUd  operates  efficiently. 

Observe  that  the  production  possibility  represented  by  (£>d,  Id>  is  the  efficiency 
frontier  point  "closest"  to  (Od,  7  ).  Here,  "close"  means  similar  in  production  technol¬ 
ogy  or  input-mix.  Thus,  the  problem  being  addressed  has  several  different  solutions,  each 
depending  on  the  input- mix  (recipe)  to  be  used  to  produce  Od* 


Since  the  value  1^,  as  determined  above,  depends  on  the  value  I  seeded,  the 
process  can  be  repeated  several  times  to  explore  the  various  alternatives  in  input-mix  for 


DMUr»  to  produce  0 


Managers  will  be  responsible  for  deciding  which  I .  (input-mix) 

a 


D  ~  - . d 

DMUq  should  use  the  following  year  to  produce  0d 


♦Feasible  set  of  inputs  and  output  values. 

16A.  Charnes  and  W.  W.  Cooper,  "The  Non- Archimedean  CCR  ratio  for  Efficiency 
Analysis:  A  Rejoinder  to  Boyd  and  Fare,"  European  Journal  of  Operational  Research, 
Vol  15  (1984),  pp  333-334. 
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To  continue  the  example,  suppose  now  that  the  data  in  Table  5  are  the  real  data 
corresponding  to  the  installations  in  MACOM  #2  for  FY86.  Then,  Table  6  now  contains 
the  results  of  evaluating  the  installation's  performance  during  FY86.  In  other  words,  now 
the  interpretation  of  results  regarding  installation  #1  is: 

•  The  efficiency  rate  assigned  to  installation  #1  in  FY86  is  60.07  percent. 

•  Installations  with  the  most  influence  on  installation  #l’s  efficiency  rate  are  4, 
12,  and  21. 

Suppose  also  that  the  managers  want  to  know  how  much  (J+K+L+M)-- 1 1  — will  cost  to 
operate  installation  #1  in  1990  assuming  that:  (1)  03,  Os,  and  I5  will  remain  at  the 

same  level  as  that  in  1986,  (2)  02  will  increase  10  percent  with  respect  to  the  1986  level, 
and  (3)  I2  (BMAR)  will  not  exceed  $5  million.  To  solve  this  problem,  the  managers 
proceed  as  explained  earlier: 

1.  Create  a  dummy  DMU  for  installation  #1  -  with:  O  =  8.904,  O  =  1.925476,  O 
=  0.693742,  I4  =  dummy  =  54.135,  I2  =  5  and  I&  =  0.596. 

2.  Add  the  above  dummy  DMU  to  the  set  of  installations  in  Table  5  and  apply  DEA 
to  the  new  set.  The  efficiency  rate  of  the  dummy  DMU  is  then  66.43  percent  (Table  7). 

3.  Decrease  the  value  of  IL  in  dummy  DMU  until  its  efficiency  rate  equals  100 
percent  (Table  7).  The  value  of  1 t  that  makes  the  dummy  DMU's  efficiency  rate  100 
percent  is  I3  =  29.31139.  This  figure  will  be  the  cost  to  operate  installation  #1  effi¬ 
ciently  in  1990  assuming  1986  prices  do  not  change. 

One  more  issue  should  be  addressed  under  this  discussion— the  reallocation  of 
resources  among  efficient  DMUs.  Here,  the  question  is  "Which  DMU  will  have  the 
greatest  potential  to  use  additional  resources,"  or,  "Which  DMUs  are  underfunded,  and  by 
how  much."  The  answer  to  these  questions  requires  a  deeper  analysis  of  the  efficiency 
frontier. 

At  first  glance,  the  underfunded  installations,  if  any,  must  be  among  the  units  rated 
100  percent  efficient  by  DEA.  As  a  general  rule  of  thumb,  resources  should  be  real¬ 
located  from  inefficient  units  to  relatively  efficient  ones.  However,  the  efficiency 
frontier  must  be  studied  closely  since  DEA  does  not  differentiate  among  relatively 
efficient  installations.  One  way  to  further  differentiate  among  relatively  efficient 
installations  using  DEA  is  by  seeding  a  small  number  of  "utopian"  installations  into  the 
set  of  DMUs  (utopian  in  the  context  of  efficiency).  By  doing  so,  only  the  fictitious 
installations  will  be  rated  relatively  efficient,  and  the  rest  of  the  DMUs  (DMU  =  j;  j  = 
l,...,n)  will  be  evaluated  against  them. 

The  main  drawback  to  this  approach  is  that  the  utopian  installations  do  not  belong 
to  the  production  possibility  set;  hence,  the  new  efficiency  rate  of  the  real  installation 
may  be  unrealistic.  However,  this  problem  can  be  alleviated  by  selecting  the  utopian 
DMU  to  be  as  realistic  as  possible. 


30 


Table  7 


Resource  Determination  With  DEA 


Ut 

U3 

U5 

V1 

V2 

V5 

°1 

°3 

°5 

*2 

'5 

+ 

+ 

S1 

S3 

S5 

S1 

S2 

S5 

0.0001 

0.0001 

95.76124 

0.842747 

3.012537 

65.96505 

Dummy  01 

8.904 

1 .925476 

0.693742 

54.135 

5 

0 

1.532 

0.376766 

0 

0 

0 

0 

REF. SET 

21 

12 

9 

LAMBDA 

0.000907 

0.174583 

2.154602 

Decreasing  l2 

0.0001 

0.351734 

123.5046 

2.339895 

0.0001 

29.63733 

Dummy  #1 

8.904 

1 .925476 

0.693767 

35.18775 

5 

0.596 

24.07107 

0 

0 

0 

0.406969 

0 

REF. SET 

21 

12 

5 

LAMBOA 

0.581312 

0.219265 

0.07808 

Further  Decreasing  l2 

0.005986 

0.323894 

143.J764 

2.713590 

0.0001 

34.32949 

Dummy  02 

8.904 

1 .925476 

0.693742 

29.31139 

5 

0.596 

0 

0 

0 

0 

0 

0 

REF. SET  Dummy  #1 

LAMBDA  1 


Oj  =  Population  served 
0^  *  Replacement  value 
Oj  =  Btu/yr . 

I,  =  J  ( J+K+L+M) 

l?  =  S8MAR 

lj  *  Sq  ft  of  shops 
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5  CONCLUSIONS  AND  RECOMMENDATIONS 


Three  modeling  techniques  have  been  studied  for  potential  in  providing  an  output- 
oriented  performance  index  to  evaluate  the  efficiency  of  RPMA  programs  at  U.S.  Army 
installations.  The  methodology  chosen  for  further  evaluation  was  DEA  because  of  its 
ability  to  handle  multiple  inputs  and  outputs  simultaneously  and  without  requiring  a  priori 
specification  of  weight  or  functional  relationships  among  these  parameters.  These 
features  make  DEA  suitable  for  application  to  not-for-profit  organizations  such  as  Army 
installations. 

The  DEA  proccess  was  analyzed,  and  it  was  concluded  that: 

1.  DMUs  identified  as  inefficient  by  DEA  were  strictly  inefficient  compared  with 
other  existing  DMUs. 

2.  DEA  can  associate  a  narrow  set  of  relatively  efficient  units  (the  reference  set) 
with  the  inefficient  unit  under  evaluation,  which  helps  managers  identify  the  source  of 
inefficiency. 

3.  DEA  identifies  an  efficiency  frontier  consisting  of  relatively  efficient  DMUs, 
which  means  this  frontier  is  a  practically  attainable  production  possibility  set. 

4.  DEA  does  not  necessarily  locate  all  inefficient  units;  thus,  some  units  rated  as 
relatively  efficient  may  be  strictly  inefficient. 

5.  DEA  identifies  alternative  paths  for  making  inefficient  DMUs  become  relatively 
efficient. 

6.  The  choice  of  input  and  output  measures  used  is  critical  to  the  inherent  value  of 
the  results. 

Input  and  output  measures  were  selected  to  fully  exploit  the  capabilities  of  DEA 
for  the  user.  A  compromise  was  reached  between  the  desired  data  and  that  which  was 
available.  Numerical  results  of  the  analysis  were  presented  to  a  panel  of  managers  for 
validation.  These  managers  suggested  that  the  model  should  further  differentiate 
between  the  installations  rated  as  100  percent  efficient. 

This  study  has  shown  that  DEA  is  a  theoretically  sound  technique  capable  of 
evaluating  the  performance  of  RPMA  at  Army  installations.  Optimal  use  of  DEA  will  be 
possible  once  the  method  is  "fine-tuned”  by  testing  alternative  input  and  output  measures 
until  its  capabilities  are  fully  exploited. 

Based  on  these  findings,  it  is  recommended  that  a  prototype  of  the  performance 
model  be  implemented  within  OORMS  to  allow  prospective  users  to  test  alternative  input 
and  output  measures. 
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APPENDIX  A: 


DATA  ENVELOPMENT  ANALYSIS-CONCEPT  AND  DERIVATION 


This  appendix  describes  the  fundamentals  of  DEA.  The  first  section  focuses  on 
mathematical  formulation  of  the  model,  with  the  second  part  summarizing  the  model's 
strengths  and  weaknesses. 


Linear  Programming  Formulation 

The  formulation  in  the  text  designated  as  Equation  2  is  an  extended  nonlinear 
programming  formulation  of  an  ordinary  fractional  programming  problem.  However, 
studies17  have  shown  that  it  can  be  transformed  into  an  equivalent  linear  programming 
problem  using  the  theory  of  linear  fractional  programming  developed  by  Charnes  and 
Cooper  as  follows: 1 8 


Maximize: 


Subject  to: 


s 

I 

r=l  ‘  ‘J  i*l 


U>e>o  r=l,..,s 
r 

V.  >  e  >  0  i  =  1,  m  [Eq  Al] 


l.  =  l  HO  . 
JO  r=l  r  rJ0 


m 

l  V.I. •  = 

i  ijo 


i=l 


m 


U  0  .  -J  V. I..  S  0;  j  *  1,  jo,  n 


The  Equation  Al  formulation  constrains  the  weighted  sum  of  the  inputs  to  be  equal 
to  1  and  maximizes  the  outputs  that  can  be  obtained.  The  other  constraints  in  Equation 
Al  transform  the  less-than-unity  constraints  of  Equation  2  (Chapter  2)  to  a  form  in  which 
the  weighted  output  cannot  exceed  the  weighted  input  for  any  DMU.  Equation  Al  is  also 
called  the  "unit  input"  formulation. 


17A.  Charnes,  W.  W.  Cooper,  and  E.  Rhodes  (1978);  A.  Charnes,  W.  W.  Cooper,  and  E. 
Rhodes  (1979). 

18 A.  Charnes  and  W.  W.  Cooper. 


The  dual  linear  program  of  Equation  A1  can  be  written  as  follows: 


m  s 

Minimize:  [0  -  e  (£  S.  +  £  S  )} 

i=l  1  r=l  r 

r  + 

Subject  to:  )  X .  0  .  -  S  =0.  r  =  1 ,  . . ,  s 

J  rJ  r  rJ° 

n 

-  V  X.  I..  -  S.  +  01..  =0  i  =  1,  m 

J  iJ  i  ijo 

X.,  S+,  S.  >  0  and  0  unrestricted  in  sign 
J  r  j 

Where: 


[Eq  A2] 


0  =  An  intensity  value  or  multiplier  of  the  observed  input  vector  XjQ 

S'  =  output  slack  for  output  V 

sT  =  input  slack  for  input  "i” 
c  =  A  small  non- Archimedean  constant. 


In  Equation  A2,  the  variable  0  is  considered  an  ’’intensity"  variable  that  reduces  the 
value  of  all  inputs  to  the  smallest  number  permitted  by  the  constraints.  The 
variables  x .  ;  j=l,..,n  are  the  dual  variables  associated  with  the  constraints  representing 
DMUjj  j=l,..,n  in  Equation  Al.  For  x ^  to  be  positive  (>0),  its  corresponding  DMUj 
constraint  in  Equation  Al  has  to  be  equal  to  0,  which  means  that  DMU=  is  in  the  refer¬ 
ence  set  of  the  DMU  being  evaluated,  DMUjq.  1 


The  name  "data  envelopment  analysis"  comes  from  the  facts  that,  in  Equation 
A2:  (1)  the  linear  combination  of  output  vectors  represented  by  the  ^  j°r  j  envelops  the 

observed  output  vector  of  the  DMU  being  evaluated  from  above,  and  (2)  the  same  linear 
combination,  but  with  the  input  vectors,  envelops  the  "minimum"  vector  of  observed 
inputs  from  below. 


The  presence  of  the  non- Archimedean  constant  in  the  objective  function  of 
Equation  A2  is  equivalent  to  a  double  optimization  since  e  is  so  small  that 
m  _  s 

e  s-  +  1  s  does  not  affect  the  value  of  0.  In  other  words,  two  optimizations 

i=l  1  r=l  r  m  _  s  + 

occur  at  the  same  time:  (1)  minimize  0  and  (2)  maximize  (£  S.  +  £  S  )  . 

• _ «  i  «  r 

1=1  r=l 


By  virtue  of  the  duality  theory  of  linear  programming: 


s  s  _  s 

Maximize  h.  =71/0.  =  Minimize  [0  ■  e  (J  S  ♦  T  S  )]  [Eq  A3] 

jo  r“j  r  rjo  i=l  r  r=l  r 

Also  note  that  S*  and  are  the  dual  variables  associated  with  the  constraints  Up  >e  , 
V|  >  c  in  Equation  A2,  respectively;  therefore,  at  optimality,  the  only  time  the  cor¬ 
responding  Up  =  e  and/or  V-  =  e  is  when  sT  >  0  and/or  S*  >  0. 
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From  the  above  discussion,  it  can  be  shown  that  a  DMU  is  considered  efficient  if 
and  only  If  e*  *  1,  and  all  S.  and  are  equal  to  zero  In  Equation  A3.  This  reason- 

ing  is  called  the  non- Archimedean  efficiency  theorem  for  which  formal  proof  can  be 
found  elsewhere. 1 9 


Efficiency  Adjustments  and  Projections 


In  addition  to  measuring  the  relative  efficiency  of  a  DMU,  DEA^provides  additional 

information  for  the  inefficient  DMU  to  become  efficient.  Let  0*,  X.,  S  . ,  S  be  an 

J  i  r 

optimal  solution  of  Equation  A2  for  DMUj_.  Then,  Equation  A2  can  be  rewritten  as: 


0*  I. 
10 


«  * 
5.  ■  >  I.  .X. 

1  j-i  lJ  J 


1—1,  ni 


0 

ro 


+*  n  * 

+  s  -  y  o  .x. 

r  j=l  rJ  J 


r*l , 


s 


[Eq  A4] 


If  the  left-hand  side  of  Equation  A4  is  replaced  with  Equation  A5: 


-* 


I-  —  0  I.  -  S .  i*l ,  ..,  m 

10  10  l  ’  ' 


"  ♦* 

0  =  0  +  S  r=l ,  . . ,  s 

ro  ro  r  '  ’ 


then  a  DMU  with  its  production  vector  represented  by  the  new  values  (0.,I.)  will  be 

1  1  iflr 

rated  efficient,  relative  to  the  same  set  DMU5;  j=l,...,n,  since  Q'*  ~  1,  S  =0  and  e.  = 

0  will  be  an  optimal  solution  of  Equation  A2.  (Formal  proof  of  this  derivation  is  shown 
elsewhere.20)  Thus,  the  transformation  (Eq  A5)  projects  DMUlV.  into  its  reference  set, 
and  the  values  S.  ,  S  and  0*  represent  the  adjustments  needed  for  DMU:  to  become 
efficient.  1  r 

To  better  explain  the  Charnes,  et  al.  formulation,  it  can  be  applied  to  a  simple 
problem  with  a  simple  geometrical  solution.  Consider  a  set  of  four  DMUs  each  with  two 
inputs,  It  and  I2,  and  one  output,  Oj.  To  simplify,  assume  that  01  is  equal  to  1  for  the 
four  DMU's.  Figure  Al  represents  the  four  DMUs  by  their  coordinate  values  for  I,  and 
I2.  For  example,  Pj  has  In  =  2,  I21  =  2,  On  =  1;  P2  has  I12  =  3,  I22  =  2,  012  =  1;  and 
so  on. 


19A.  Charnes  and  W.  W.  Cooper,  "Preface  to  Topics  in  Data  Envelopment  Analysis," 
Annals  of  Operations  Research,  Vol  2  (198S),  pp  59-94. 

20 A.  Charnes  and  W.  W.  Cooper  (1985). 
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Figure  Al.  Example  problem  with  a  simple  geometrical  solution. 


To  evaluate  P2,  its  coordinates  are  inserted  into  Equation  A3  to  obtain: 
Minimize:  0  -  eS~  -  eS2  -  eS* 

Subject  to:  39  =  2X^  ♦  3X2  +  lx^  +  6x^  +  S~ 

20  =  2Xl  *  2\2  +  4X3  +  4X4  +  S~ 

1  *  S1  ■  X1  *  k2  *  *3  *  k4 

for  which  the  solution  is: 


'ff  —  -fc  it 

0  =  1 ,  =1,  Xj  =1  and 


Although  9*  =  1,  DMU  P2  is  not  fully  efficient  since  S  *  equals  1.  The  fact  that 
X^  =  1  means  that  P2  is  being  evaluated  against  Pt.  1  Moreover,  the  value  of 

S3  =  1  means  that,  when  comparing  P2  with  Pt  ,  P2  uses  one  more  unit  of  input  1  than 
does  P:  to  produce  the  same  amount  of  output.  In  other  words,  Pt  produces  the  same 
amount  of  output  as  P2,  but  with  one  unit  less  of  input,  It,  and  the  same  amount  of  I2. 

For  P2  to  be  as  efficient  as  Plf  it  has  to  reduce  Ij  by  one  unit,  which  produces  the 
same  result  as  if  the  transformation  (Eq  A5)  had  been  applied. 


The  solution  of  Equation  A3  for  P  l  is: 

*  -*  -*  +*  * 
0  =  1,  Sx  =  S2  =  Sx  =0,  X1 


1 


*  *  * 

X2  X3  X4  * 

which  means  that  P  is  fully  efficient,  and  also  that  it  is  evaluated  against  itself, 
★  1 

(  x^  =  1).  Thus,  no  other  DMU  is  more  efficient  than  Pj. 

In  the  same  way,  the  result  of  evaluating  ?k  is: 


1 


*  *  * 


which  means  that  P4  is  fully  efficient,  and  no  other  DMU  is  more  efficient  than 
P4,  (x4  =  1) . 

Referring  to  Figure  Al,  the  solid  line  connecting  P^,  P^  and  P2  is  called  the 
"efficiency  frontier"  and  is  made  up  of  convex  combinations  of  DMU  with  the  efficiency 
rate  equal  to  1,  (o  =  1).  Note  that,  although  P2  is  a  frontier  point,  it  is  not  DEA 
efficient  because  (Sx  =1).  A  point  such  as  P2  is  called,  in  DEA  jargon,  a  "corner 
point"  of  the  efficiency  frontier. 

Similarly,  the  result  of  evaluating  P3  is: 

0*  =  1/2  S~*  =  1,  X  *  =  1  and 


=  0 


Ar  ^ 

which  is  to  say  that  P3  is  being  evaluated  against  P  =  l(x1  =  1).  The  value  of  e  =1/2 
represents  the  ratio  of  the  Euclidean  distance  from  the  origin  to  P3.  For  P3  to  be  as 
efficient  as  P  ,  it  has  to  reduce  its  inputs  in  accordance  with  Equation  A5  as  follows: 


Il3  =  6  x  1/2  -  1  =  2 
I23  =  4  x  1/2  =  2 

Ar 

The  reduction  in  inputs  represented  by  0  bring  P3  to  P2.  This  reduction  is  a 

contraction  in  which  the  mix  of  inputs  (technology)  does  not  change.  The  reduction  in 

* 

input  represented  by  S.  ,  on  the  contrary,  requires  a  change  in  input  mix,  which  brings 

^  At 

P3  from  P2  to  P  ,  and  it  does  not  increase  the  efficiency  rate,  0  .  Consequently,  the 
contraction  represented  by  0  1^  projects  the  inefficient  unit  into  the  efficiency  fron¬ 

tier,  whereas  the  reduction  represented  by  brings  the  inefficient  unit  into  the  so- 
called  DEA  efficiency  region. 


39 


DEA  Properites  and  Interpretations 

Properties  and  interpretations  of  the  DEA  model  above  are  summarized  below. 
Relative  Efficiency 

One  hundred  percent  relative  efficiency  is  attained  by  any  DMU  only  when 
comparison  with  other  relevant  DMUs  provides  no  evidence  of  inefficiency  in  the  use  of 
any  input  or  output. 

Weights 

DEA  does  not  require  a  priori  determination  of  weight  to  compute  efficiency. 
Efficiency  Rating  (0  ) 

The  efficiency  rating  assigned  by  DEA  is  the  best  possible  value  attainable  by  the 
DMU  being  evaluated.  The  efficiency  rating  does  not  depend  on  the  units  in  which  the 
inputs  and  outputs  are  measured. 

Reference  Set 

DEA  also  provides  the  relatively  inefficient  units  with  a  set  of  relatively  efficient 
units  that  have  a  "similar"  input/output  mix. 

Efficiency  Frontier 

The  efficiency  frontier  consists  of  a  piecewise  linear  combination  of  the  efficient 
units,  which  means  that  it  is  practically  attainable  since  it  is  in  the  production  possibility 
set. 

Efficiency  Improvement 

'ff  •ff 

V.  and  U  are  the  dual  variables  of  Equation  A2,  representing  the  marginal  gain  in 
efficiency  if  one  less  unit  of  the  ith  resource  is  used  or  one  more  unit  of  the  rth  output  is 
produced. 

The  transformation  (Eq  A5)  that  makes  an  inefficient  unit  DEA  efficient  can  be 
seen  as  a  two-stage  process: 

•ff 

1.  The  reduction  of  Ij  to  0  I.  increases  the  efficiency  rate  to  1  without  changing 
the  input  mix  ("recipe"). 

0  ^  +★ 

2.  and  represent  the  additional  marginal  improvements  possible  for  the 
unit  to  become  truly  efficient. 

Production  Process 

The  DEA  model  assumes  that  each  input  has  some  relationship  to  one  or  more 
outputs,  but  it  is  not  necessary  to  specify  these  functional  relationships  explicitly. 
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APPENDIX  B: 


EXPLANATION  OF  COMPUTER  OUTPUT 


Table  B1  contains  the  FY85  computer  output  for  the  21  installations  of  MACOM 
#1.  For  each  installation,  there  are  five  lines,  of  which  the  first  three  lines  contain  the 
values  of  the  weight,  slack,  and  input/output  as  follows: 

Column  1:  Name  of  the  installation. 

Columns  2-6:  Output-related  values  (note  that  02  is  missing  because  of  data 
unavailability).  For  the  installation,  there  are  three  numbers  under  each  of  these 
columns.  Consider  for  example,  column  2  (Output-1:  Population  Served)  of  row  1 
(installation  #1): 

1.  First  number  (ut)  =  0.001163.  This  is  the  weight  assigned  to  output  Oj  (popula¬ 
tion  size)  installation  #1. 

2.  Second  number  (O^  =  50.79.  This  is  the  value  of  output  Ol  for  installation  #1. 
That  is,  the  population  of  installation  #1  is  50,790. 

3.  Third  number  (SO  )  =  0.  This  is  the  slack  in  output  1.  To  illustrate  this  value, 
consider  installation  #5  wnich  has  5.065528  thousand  slack.  Thus,  using  the  current 
resources,  installation  #5  could  serve  roughly  5065  additional  people.  (Appendix  A  gives 
details.) 

Columns  7-11:  Input-related  values.  For  example,  column  7  (Input  2:  K  account)  of 
row  1  has  three  numbers: 

1.  V2  =  0:  the  weight  assigned  to  K  account  by  installation  #1. 

2.  I2  =  19.11532:  the  amount  of  funds  (in  millions  of  dollars)  received  by  installa¬ 
tion  #1  under  K  account. 

3.  SI2  =  0:  Slack  (inefficiently  used)  money  by  installation  K  under  K  account. 

Column  12:  Efficiency  rating  (performance)  of  the  installation.  This  efficiency 
rating  is  relative  to  other  installations  in  the  reference  set. 

In  addition,  the  reference  set  of  each  inefficient  installation  and  its  corresponding 
values  (as  described  in  Appendix  A)  are  reported  in  the  two  lines  named  REF.  SET.  and 

LAMBDA. 

Tables  B2  and  B3  contain  the  results  for  MACOM  #1  for  FY84  and  FY83,  respect¬ 
ively.  Table  B4  contains  the  results  for  MACOM  #2  for  FY85;  Table  B5  lists  results  of 
the  joint  analysis  for  MACOMs  #1  and  #2  in  FY85. 
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DEA  Results  for  MACOM  #1,  FY85 
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